Dirichlet Process Mixture Model Research Articles

ScDAC: deep adaptive clustering of single-cell transcriptomic data with coupled autoencoder and dirichlet process mixture model.

Clustering analysis for single-cell RNA sequencing (scRNA-seq) data is an important step in revealing cellular heterogeneity. Many clustering methods have been proposed to discover heterogenous cell types from scRNA-seq data. However, adaptive clustering with accurate cluster number reflecting intrinsic biology nature from large-scale scRNA-seq data remains quite challenging. Here we propose a single-cell Deep Adaptive Clustering (scDAC) model by coupling the Autoencoder (AE) and the Dirichlet Process Mixture Model (DPMM). By jointly optimizing the model parameters of AE and DPMM, scDAC achieves adaptive clustering with accurate cluster numbers on scRNA-seq data. We verify the performance of scDAC on five subsampled datasets with different numbers of cell types and compare it with fifteen widely-used clustering methods across nine scRNA-seq datasets. Our results demonstrate that scDAC can adaptively find accurate numbers of cell types or subtypes and outperforms other methods. Moreover, the performance of scDAC is robust to hyperparameter changes. The scDAC is implemented in Python. The source code is available at https://github.com/labomics/scDAC. Supplementary data are available at Bioinformatics online.

High accuracy ranging for space debris with spaceborne single photon Lidar.

The increasing risk posed by space debris highlights the need for accurate localization techniques. Spaceborne single photon Lidar (SSPL) offers a promising solution, overcoming the limitations of traditional ground-based systems by providing expansive coverage and superior maneuverability without being hindered by weather, time, or geographic constraints. This study introduces a novel approach leveraging non-parametric Bayesian inference and the Dirichlet process mixture model (DPMM) to accurately determine the distance of space debris in low Earth orbit (LEO), where debris exhibits nonlinear, high dynamic motion characteristics. By integrating extended Kalman filtering (EKF) for range gating, our method captures the temporal distribution of reflected photons, employing Markov chain Monte Carlo (MCMC) for iterative solutions. Experimental outcomes demonstrate our method's superior accuracy over conventional statistical techniques, establishing a clear correlation between radial absolute velocity and ranging error, thus significantly enhancing monostatic space debris localization.

Bayesian collapsed Gibbs sampling for a stochastic volatility model with a Dirichlet process mixture

SummaryThis paper replicates the results of the stochastic volatility–Dirichlet process mixture (SV‐DPM) models proposed in Jensen and Maheu (2010) in both a narrow and a wide sense. By using a normal‐Wishart prior and the collapsed Gibbs sampling method, our algorithm can be applied for more general settings, and it is more efficient for sampling the Dirichlet process mixture. For the stochastic volatility component, we adopt the method in Chan (2017) to further increase the overall efficiency of our algorithm. Using the same dataset, we obtain mixed results. Some of the results have significant differences. If we use recent time period dataset, which includes the COVID‐19 pandemic period, the log market portfolio volatility seems to increase in terms of the number of clusters and size of magnitude.

A Dirichlet process mixture regression model for the analysis of competing risk events

We develop a regression model for the analysis of competing risk events. The joint distribution of the time to these events is flexibly characterized by a random effect which follows a discrete probability distribution drawn from a Dirichlet Process, explaining their variability. This entails an additional layer of flexibility of this joint model, whose inference is robust with respect to the misspecification of the distribution of the random effects. The model is analysed in a fully Bayesian setting, yielding a flexible Dirichlet Process Mixture model for the joint distribution of the time to events. An efficient MCMC sampler is developed for inference. The modelling approach is applied to the empirical analysis of the surrending risk in a US life insurance portfolio previously analysed by Milhaud and Dutang (2018). The approach yields an improved predictive performance of the surrending rates.

A high‐dimensional multinomial logit model

SummaryThe number of parameters in a standard multinomial logit model increases linearly with the number of choice alternatives and number of explanatory variables. Because many modern applications involve large choice sets with categorical explanatory variables, which enter the model as large sets of binary dummies, the number of parameters in a multinomial logit model is often large. This paper proposes a new method for data‐driven two‐way parameter clustering over outcome categories and explanatory dummy categories in a multinomial logit model. A Bayesian Dirichlet process mixture model encourages parameters to cluster over the categories, which reduces the number of unique model parameters and provides interpretable clusters of categories. In an empirical application, we estimate the holiday preferences of 11 household types over 49 holiday destinations and identify a small number of household segments with different preferences across clusters of holiday destinations.

Analysis of Censored Data Using Dirichlet Process Mixture Model with Generalized Inverse Weibull Distribution as Kernel

Analysis of Censored Data Using Dirichlet Process Mixture Model with Generalized Inverse Weibull Distribution as Kernel

Dirichlet process mixture models to impute missing predictor data in counterfactual prediction models: an application to predict optimal type 2 diabetes therapy

BackgroundThe handling of missing data is a challenge for inference and regression modelling. A particular challenge is dealing with missing predictor information, particularly when trying to build and make predictions from models for use in clinical practice.MethodsWe utilise a flexible Bayesian approach for handling missing predictor information in regression models. This provides practitioners with full posterior predictive distributions for both the missing predictor information (conditional on the observed predictors) and the outcome-of-interest. We apply this approach to a previously proposed counterfactual treatment selection model for type 2 diabetes second-line therapies. Our approach combines a regression model and a Dirichlet process mixture model (DPMM), where the former defines the treatment selection model, and the latter provides a flexible way to model the joint distribution of the predictors.ResultsWe show that DPMMs can model complex relationships between predictor variables and can provide powerful means of fitting models to incomplete data (under missing-completely-at-random and missing-at-random assumptions). This framework ensures that the posterior distribution for the parameters and the conditional average treatment effect estimates automatically reflect the additional uncertainties associated with missing data due to the hierarchical model structure. We also demonstrate that in the presence of multiple missing predictors, the DPMM model can be used to explore which variable(s), if collected, could provide the most additional information about the likely outcome.ConclusionsWhen developing clinical prediction models, DPMMs offer a flexible way to model complex covariate structures and handle missing predictor information. DPMM-based counterfactual prediction models can also provide additional information to support clinical decision-making, including allowing predictions with appropriate uncertainty to be made for individuals with incomplete predictor data.

Supervision and early warning of abnormal data in Internet of Things based on unsupervised attention learning

With the continuous development of artificial intelligence and big data, more and more communication protocols have appeared in the network connection. Because the network protocol content specification should not need to obtain the content of the common protocol by any means, it faces different attacks, which increases the complexity of network security detection to an unlimited extent. The absolute security analysis of the content of the relevant data protocol is not too limited. Based on the content of the unknown binary number protocol in the Internet of Things, this paper does the reverse research. To address the inefficiency, learning is focused on the inverse aspect of the content processing logic of the protocol for unsupervised execution attention. This paper proposes a method based on the name of the Dirichlet process mixture model normal field and the unknown protocol of the Internet of Things. It also establishes the behavioral criteria that process the logical inference least squares method to infer the content of the target protocol. Based on the original external characteristics of abnormal data in the Internet of Things, the neural network model of attention mechanism and the model checking of variational self-encoder are chosen. An innovative multi-dimensional spatial neural network model with a centralized control mechanism is taken to extract the spatial relevant information from the original external features, which are regarded as the intermediate external features. The intermediate extrinsic features are input to a variational self-encoder for computation and reconstruction. The experimental results verify that the time required by the proposed model is 45% less than that of the traditional structure. The flow processing improves by 36.7% compared with the optimization of the parameter control function. The specific method only relies on the original external characteristics of the Internet of Things traffic and can achieve better data anomaly detection and classification under the condition of not significantly improving the detection time. The abnormal data detection scheme of the Internet of Things designed in this paper not only fills the detection gap of the current mainstream Internet of Things system which lacks an autonomous supervision mechanism, but also effectively blocks external network intrusion and further improves the management authority verification system. It plays an indispensable role in the long-term stable operation of the Internet of Things platform.

Context-Based Meta-Reinforcement Learning with Bayesian Nonparametric Models.

Deep reinforcement learning agents usually need to collect a large number of interactions to solve a single task. In contrast, meta-reinforcement learning (meta-RL) aims to quickly adapt to new tasks using a small amount of experience by leveraging the knowledge from training on a set of similar tasks. State-of-the-art context-based meta-RL algorithms use the context to encode the task information and train a policy conditioned on the inferred latent task encoding. However, most recent works are limited to parametric tasks, where a handful of variables control the full variation in the task distribution, and also failed to work in non-stationary environments due to the few-shot adaptation setting. To address those limitations, we propose MEta-reinforcement Learning with Task Self-discovery (MELTS), which adaptively learns qualitatively different nonparametric tasks and adapts to new tasks in a zero-shot manner. We introduce a novel deep clustering framework (DPMM-VAE) based on an infinite mixture of Gaussians, which combines the Dirichlet process mixture model (DPMM) and the variational autoencoder (VAE), to simultaneously learn task representations and cluster the tasks in a self-adaptive way. Integrating DPMM-VAE into MELTS enables it to adaptively discover the multi-modal structure of the nonparametric task distribution, which previous methods using isotropic Gaussian random variables cannot model. In addition, we propose a zero-shot adaptation mechanism and a recurrence-based context encoding strategy to improve the data efficiency and make our algorithm applicable in non-stationary environments. On various continuous control tasks with both parametric and nonparametric variations, our algorithm produces a more structured and self-adaptive task latent space and also achieves superior sample efficiency and asymptotic performance compared with state-of-the-art meta-RL algorithms.

Data association load uncertainty and risk aversion in electricity markets with data center participation in the demand response

Recently, demand response (DR) programs are being promoted in the power market to strengthen the role played by the load side in the power market and to increase the flexibility of loaders to participate in the power market. Data centers are uniquely positioned to participate as large-scale power users in the power market operation process. However, the issue of load uncertainty risk is not adequately considered in recent DR programs for data center participation in the power market. Regarding this issue, we propose a framework for facilitating bids for data center participation in DR programs in the power market. In the proposed framework, the data association Dirichlet process mixture model (DA-DPMM) is introduced to characterize the load uncertainty, and the association of load data is investigated to obtain higher prediction accuracy. To further reduce the uncertainty risk, we propose a two-stage market cleaning model based on the conditional value at risk (CVaR). We also analyze the CVaR, the social welfare derived from the Gaussian process model (GM), the Dirichlet process mixture model (DPMM), and the DA-DPMM. The results demonstrate the superiority of the DA-DPMM approach in a two-stage market cleaning model based on CVaR.

Comparative Analysis of Improved Dirichlet Process Mixture Model

Due to the development of information technology, large amounts of data are generated every day in various industries such as engineering, healthcare, finance, anomaly detection, image recognition, and artificial intelligence. This massive data poses the challenge of analyzing accurately and appropriate classifications. The traditional clustering methods require specifying the number of clusters and are mostly based on distance, which cannot effectively consider the correlations between different indicators of high-dimensional and multi-source data. Moreover, the number of clusters cannot automatically adjust when new data is generated. In order to improve the clustering analysis of high-dimensional and multi-source data in a big data environment, this study utilizes non-parametric mixture models based on distribution clustering, which does not require specifying the number of clusters and can auto update with the data. By combining Principal Component Analysis (PCA), t-Distributed Stochastic Neighbour Embedding (t-SNE), and the non-parametric Bayesian method called Dirichlet Process Mixture Model (DPMM), the Bayesian non-parametric PCA model (PCA-DPMM) and Bayesian non-parametric t-SNE model (TSNE-DPMM) are proposed. The Chinese restaurant process of DPMM is used for sampling by introducing a finite normal mixture distribution. The clustering results on the iris dataset are compared and analyzed. The accuracy of DPMM and TSNE-DPMM reaches 0.97, while PCA-DPMM achieves a maximum accuracy of only 0.94. When different numbers of iterations are set, TSNE-DPMM maintains an accuracy ranging from 0.92 to 0.97, DPMM ranges from 0.66 to 0.97, and PCA-DPMM ranges from 0.73 to 0.94. Therefore, the proposed TSNE-DPMM ensures accuracy and exhibits better model stability in clustering results. Future research can explore the improvement of the model by incorporating deep learning algorithms, among others, to further enhance its performance. Additionally, applying the TSNE-DPMM model to data analysis in other fields is also a future research direction. Through these efforts, we can better tackle the challenges of analyzing high-dimensional and multi-source data in a big data environment and extract valuable information from it.

Variational inference for semiparametric Bayesian novelty detection in large datasets

AbstractAfter being trained on a fully-labeled training set, where the observations are grouped into a certain number of known classes, novelty detection methods aim to classify the instances of an unlabeled test set while allowing for the presence of previously unseen classes. These models are valuable in many areas, ranging from social network and food adulteration analyses to biology, where an evolving population may be present. In this paper, we focus on a two-stage Bayesian semiparametric novelty detector, also known as Brand, recently introduced in the literature. Leveraging on a model-based mixture representation, Brand allows clustering the test observations into known training terms or a single novelty term. Furthermore, the novelty term is modeled with a Dirichlet Process mixture model to flexibly capture any departure from the known patterns. Brand was originally estimated using MCMC schemes, which are prohibitively costly when applied to high-dimensional data. To scale up Brand applicability to large datasets, we propose to resort to a variational Bayes approach, providing an efficient algorithm for posterior approximation. We demonstrate a significant gain in efficiency and excellent classification performance with thorough simulation studies. Finally, to showcase its applicability, we perform a novelty detection analysis using the openly-available dataset, a large collection of satellite imaging spectra, to search for novel soil types.

Spatially‐correlated time series clustering using location‐dependent Dirichlet process mixture model

AbstractThe Dirichlet process mixture (DPM) model has been widely used as a Bayesian nonparametric model for clustering. However, the exchangeability assumption of the Dirichlet process is not valid for clustering spatially correlated time series as these data are indexed spatially and temporally. While analyzing spatially correlated time series, correlations between observations at proximal times and locations must be appropriately considered. In this study, we propose a location‐dependent DPM model by extending the traditional DPM model for clustering spatially correlated time series. We model the temporal pattern as an infinite mixture of Gaussian processes while considering spatial dependency using a location‐dependent Dirichlet process prior over mixture components. This encourages the assignment of observations from proximal locations to the same cluster. By contrast, because mixture atoms for modeling temporal patterns are shared across space, observations with similar temporal patterns can be still grouped together even if they are located far apart. The proposed model also allows the number of clusters to be automatically determined in the clustering procedure. We validate the proposed model using simulated examples. Moreover, in a real case study, we cluster adjacent roads based on their traffic speed patterns that have changed as a result of a traffic accident occurred in Seoul, South Korea.

Identifying Subpopulations of Cells in Single-Cell Transcriptomic Data: A Bayesian Mixture Modeling Approach to Zero Inflation of Counts.

In the study of single-cell RNA-seq (scRNA-Seq) data, a key component of the analysis is to identify subpopulations of cells in the data. A variety of approaches to this have been considered, and although many machine learning-based methods have been developed, these rarely give an estimate of uncertainty in the cluster assignment. To allow for this, probabilistic models have been developed, but scRNA-Seq data exhibit a phenomenon known as dropout, whereby a large proportion of the observed read counts are zero. This poses challenges in developing probabilistic models that appropriately model the data. We develop a novel Dirichlet process mixture model that employs both a mixture at the cell level to model multiple populations of cells and a zero-inflated negative binomial mixture of counts at the transcript level. By taking a Bayesian approach, we are able to model the expression of genes within clusters, and to quantify uncertainty in cluster assignments. It is shown that this approach outperforms previous approaches that applied multinomial distributions to model scRNA-Seq counts and negative binomial models that do not take into account zero inflation. Applied to a publicly available data set of scRNA-Seq counts of multiple cell types from the mouse cortex and hippocampus, we demonstrate how our approach can be used to distinguish subpopulations of cells as clusters in the data, and to identify gene sets that are indicative of membership of a subpopulation.

Quantifying the uncertainty of partitions for infinite mixture models

Bayesian clustering models, such as Dirichlet process mixture models (DPMMs), are sophisticated flexible models. They induce a posterior distribution on the set of all partitions of a set of observations. Analysing this posterior distribution is of great interest, but it comes with several challenges. First of all, the number of partitions is overwhelmingly large even for moderate values of the number of observations. Consequently the sample space of the posterior distribution of the partitions is not explored well by MCMC samplers. Second, due to the complexity of representing the uncertainty of partitions, usually only maximum a posteriori estimates of the posterior distribution of partitions are provided and discussed in the literature. In this paper we propose a numerical and graphical method for quantifying the uncertainty of the clusters of a given partition of the data and we suggest how this tool can be used to learn about the partition uncertainty.

Full uncertainty analysis for Bayesian nonparametric mixture models

A full posterior analysis for nonparametric mixture models using Gibbs-type prior distributions is presented. This includes the well known Dirichlet process mixture (DPM) model. The random mixing distribution is removed enabling a simple-to-implement Markov chain Monte Carlo (MCMC) algorithm. The removal procedure takes away some of the posterior uncertainty and how it is replaced forms a novel aspect to the work. The removal, MCMC algorithm and replacement of the uncertainty only require the probabilities of a new or an old value associated with the corresponding Gibbs-type exchangeable sequence. Consequently, no explicit representations of the prior or posterior are required and instead only knowledge of the exchangeable sequence is needed. This allows the implementation of mixture models with full posterior uncertainty, not previously possible, including one introduced by Gnedin. Numerous illustrations are presented, as is an R-package called CopRe which implements the methodology, and other supplemental material.

Bayesian prognosis analysis of human papillomavirus-associated head and neck cancer using hierarchical Dirichlet process mixture models

The incidence and prognosis of Head and Neck Cancer (HNC) depend heavily on patients’ Human PapillomaVirus (HPV) status. Prognosis analysis of HPV-associated HNC is of clinical importance because in-depth understanding of the survival distribution is valuable for designing more informed treatment strategies. In this article, we develop a novel Hierarchical Dirichlet Process Weibull Mixture Model (HDP-WMM) to study the prognosis of HNC patients given their HPV status. The HDP-WMM is capable of simultaneously characterizing the survival distributions of grouped data and capturing the dependence among different groups. Moreover, the HDP-WMM can identify clusters of patients based on their outcomes, providing additional information for exploring patient subtypes. Effective Markov chain Monte Carlo sampling algorithms are designed for model inference and function estimation. The clustering structure is identified by summarizing the posterior samples of the data random partition using the Bayesian cluster analysis tool. A simulation study is designed to validate the performance of the proposed inference methods. The practical utility of the proposed HDP-WMM is demonstrated by a case study on prognosis analysis of HPV-associated HNC. Our results show that the Bayesian HDP-WMM achieves satisfactory performance on estimating the survival functions and clustering patients based on their outcomes.

Bayesian Joint Modeling Analysis of Longitudinal Proportional and Survival Data

This paper focuses on a joint model to analyze longitudinal proportional and survival data. We utilize a logit transformation on the longitudinal proportional data and employ a partially linear mixed-effect model. With this model, we estimate the unknown function of time using the B-splines technique. Additionally, we introduce a centered Dirichlet process mixture model (CDPMM) to capture the random effects, allowing for a flexible distribution. The survival data are assumed using a Cox proportional hazard model, and the sharing random effects joint model is developed for the two types of data. We develop a Bayesian Lasso (BLasso) approach that combines the Gibbs sampler and the Metropolis–Hastings algorithm. The proposed method allows for the estimation of unknown parameters and the selection of significant covariates simultaneously. We evaluate the performance of our proposed methods through simulation studies and also provide an illustration of our methodologies using an example from the MA.5 research experiment.

Microblog topic detection using evolutionary clustering and social network information

SummaryThe messages in microblogs, like Twitter, present a wealth of information around the hot topics and public opinions. Topic detection and tracking enables a better understanding of the incoming information from social networks. Many of the studies performed in this field consider a set of predefined number of topics (clusters) which remain constant during the detection process. Such approaches are not efficient when data is dynamic and cumulative. In addition, nonparametric evolutionary topic models do not have an accurate performance on short texts due to the data deficiency. In this article, we propose an evolutionary clustering model based on the Bayesian nonparametric Dirichlet process mixture model, which automatically determines the number of clusters. To improve the algorithm performance when dealing with short social media texts, temporal information of tweets and other social network parameters are used in a weighted combination to augment the text similarity in detecting topics. Model evaluation is performed on the data gathered from the Twitter during a 2.5‐month period. Results reveal that the social network information can promote the topic detection performance. The proposed model outperforms previous study in terms of topic coherence and clustering performance, leading to a better clustering on short texts.

Dirichlet process mixture models for the analysis of repeated attempt designs.

In longitudinal studies, it is not uncommon to make multiple attempts to collect a measurement after baseline. Recording whether these attempts are successful provides useful information for the purposes of assessing missing data assumptions. This is because measurements from subjects who provide the data after numerous failed attempts may differ from those who provide the measurement after fewer attempts. Previous models for these designs were parametric and/or did not allow sensitivity analysis. For the former, there are always concerns about model misspecification and for the latter, sensitivity analysis is essential when conducting inference in the presence of missing data. Here, we propose a new approach which minimizes issues with model misspecification by using Bayesian nonparametrics for the observed data distribution. We also introduce a novel approach for identification and sensitivity analysis. We re-analyze the repeated attempts data from a clinical trial involving patients with severe mental illness and conduct simulations to better understand the properties of ourapproach.